Natural language processor for using speech to cognitively detect and analyze deviations from a baseline

ABSTRACT

A natural language processing system for analyzing speech includes a computer processing device configured to receive recorded speech of a person. The computer processing device constructs a baseline speech model of the person, the baseline speech model of the person including a property of speech based on a personal attribute of the person, compares current recorded speech of the person to the baseline speech model of the person to determine a deviation of the property of speech therebetween, and determines if the deviation of the property of speech meets a threshold of the property of speech that is defined for a disorder.

BACKGROUND

The present invention generally relates to computing systems thatperform natural language processing, and more specifically, to computingsystems that use natural language processing (NLP) to evaluate aperson's speech over time, cognitively detect subtle variations of theperson's speech, and utilize the cognitively detected speech variationsto aid in determining whether the person has a cognitive disorder.

NLP generally identifies a suite of computer-based tools that draw onthe fields of computer science, artificial intelligence, andcomputational linguistics to manage the interactions between computersand humans using language (i.e., natural language). As such, NLP systemsare related to the area of human-computer interaction. Among thechallenges in implementing NLP systems is enabling computers to derivemeaning from NL inputs (e.g., speech), as well as the effective andefficient generation of NL outputs.

SUMMARY

Embodiments of the present invention are directed to a natural languageprocessing system for analyzing speech. A non-limiting example of thesystem includes a computer processing device configured to receiverecorded speech of a person. The computer processing device constructs abaseline speech model of the person, the baseline speech model of theperson including a property of speech based on a personal attribute ofthe person, compares current recorded speech of the person to thebaseline speech model of the person to determine a deviation of theproperty of speech therebetween, and determines if the deviation of theproperty of speech meets a threshold of the property of speech that isdefined for a disorder.

Embodiments of the present invention are directed to acomputer-implemented method that processes natural language foranalyzing speech. A non-limiting example of the computer-implementedmethod includes receiving recorded speech of a person, constructing abaseline speech model of the person, the baseline speech model of theperson including a property of speech based on a personal attribute ofthe person, comparing current recorded speech of the person to thebaseline speech model of the person to determine a deviation of theproperty of speech therebetween, and determining if the deviation of theproperty of speech meets a threshold of the property of speech that isdefined for a disorder.

Embodiments of the invention are directed to a computer program productwhich processes natural language for analyzing speech. A non-limitingexample of the computer program product includes a computer readablestorage medium having program instructions embodied therewith, where theprogram instructions are executable by a computer to cause the computerto perform a method. The method includes receiving recorded speech of aperson, constructing a baseline speech model of the person, the baselinespeech model of the person including a property of speech based on apersonal attribute of the person, comparing current recorded speech ofthe person to the baseline speech model of the person to determine adeviation of the property of speech therebetween, and determining if thedeviation of the property of speech meets a threshold of the property ofspeech that is defined for a disorder.

Additional technical features and benefits are realized through thetechniques of the present invention. Embodiments and aspects of theinvention are described in detail herein and are considered a part ofthe claimed subject matter. For a better understanding, refer to thedetailed description and to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The specifics of the exclusive rights described herein are particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features and advantages ofthe embodiments of the invention are apparent from the followingdetailed description taken in conjunction with the accompanying drawingsin which:

FIG. 1 depicts a system of speech detection and analysis according to anembodiment of the invention;

FIG. 2 depicts a flowchart of a method speech detection and analysisaccording to an embodiment of the invention; and

FIG. 3 depicts major hardware of a processing device according to anembodiment of the invention.

The diagrams depicted herein are illustrative. There can be manyvariations to the diagram or the operations described therein withoutdeparting from the spirit of the invention. For instance, the actionscan be performed in a differing order or actions can be added, deletedor modified. Also, the term “coupled” and variations thereof describeshaving a communications path between two elements and does not imply adirect connection between the elements with no interveningelements/connections between them. All of these variations areconsidered a part of the specification.

In the accompanying figures and following detailed description of thedisclosed embodiments, the various elements illustrated in the figuresare provided with two or three digit reference numbers. With minorexceptions, the leftmost digit(s) of each reference number correspond tothe figure in which its element is first illustrated.

DETAILED DESCRIPTION

Various embodiments of the invention are described herein with referenceto the related drawings. Alternative embodiments of the invention can bedevised without departing from the scope of this invention. Variousconnections and positional relationships (e.g., over, below, adjacent,etc.) are set forth between elements in the following description and inthe drawings. These connections and/or positional relationships, unlessspecified otherwise, can be direct or indirect, and the presentinvention is not intended to be limiting in this respect. Accordingly, acoupling of entities can refer to either a direct or an indirectcoupling, and a positional relationship between entities can be a director indirect positional relationship. Moreover, the various tasks andprocess steps described herein can be incorporated into a morecomprehensive procedure or process having additional steps orfunctionality not described in detail herein.

The following definitions and abbreviations are to be used for theinterpretation of the claims and the specification. As used herein, theterms “comprises,” “comprising,” “includes,” “including,” “has,”“having,” “contains” or “containing,” or any other variation thereof,are intended to cover a non-exclusive inclusion. For example, acomposition, a mixture, process, method, article, or apparatus thatcomprises a list of elements is not necessarily limited to only thoseelements but can include other elements not expressly listed or inherentto such composition, mixture, process, method, article, or apparatus.

Additionally, the term “exemplary” is used herein to mean “serving as anexample, instance or illustration.” Any embodiment or design describedherein as “exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments or designs. The terms “at least one”and “one or more” may be understood to include any integer numbergreater than or equal to one, i.e. one, two, three, four, etc. The terms“a plurality” may be understood to include any integer number greaterthan or equal to two, i.e. two, three, four, five, etc. The term“connection” may include both an indirect “connection” and a direct“connection.”

The terms “about,” “substantially,” “approximately,” and variationsthereof, are intended to include the degree of error associated withmeasurement of the particular quantity based upon the equipmentavailable at the time of filing the application. For example, “about”can include a range of ±8% or 5%, or 2% of a given value.

For the sake of brevity, conventional techniques related to making andusing aspects of the invention may or may not be described in detailherein. In particular, various aspects of computing systems and specificcomputer programs to implement the various technical features describedherein are well known. Accordingly, in the interest of brevity, manyconventional implementation details are only mentioned briefly herein orare omitted entirely without providing the well-known system and/orprocess details.

Turning now to an overview of technologies that are more specificallyrelevant to aspects of the invention, NLP question & answer (NLP Q&A)systems answer NL questions by querying data repositories and applyingelements of language processing, information retrieval and machinelearning to arrive at a conclusion. NLP Q&A systems assist humans withcertain types of semantic query and search operations, such as the typeof natural question-and-answer paradigm of a medical environment. Anexample NLP Q&A system is IBM's DeepQA technologies, which are able tounderstand complex questions input to the system in natural language,and are able to answer the questions to augment human handling of thesame questions within a given environment, such as a medical inquiry anddiagnostic paradigm.

An anatomical or physiological characteristic of a person can bemeasured in the person's speech. Voice is generally considered sounduttered by the mouth, especially that uttered by human beings in speechwhere the sound possesses some property or characteristic. Speech isgenerally considered the faculty of uttering articulate sounds or wordssuch as the ability to speak or to use vocalizations to communicate.Hereinafter, “voice” and “speech” may be used interchangeably unlessotherwise contradicted within the context of the reference thereto.

Quantitative and qualitative properties and characteristic of a person'svoice or speech can include, but are not limited to cadence, tone,pitch, speed, word content, word complexity, pauses, periodicity,volume, semantic structure, sound produced during speech, coherency,etc. The existence of or a progression in a property or characteristicof a person's voice or speech over time can be expected due to factorssuch as aging, tooth loss or compromised dentition, speech impedimentssuch as stuttering, etc.

The existence, absence, change or progression of a property orcharacteristic in a person's speech outside of what is expected, can bean early signature of a brain related illness or injury or aneurological disorder. For example, aphasia and verbal apraxia aresymptoms of stroke. As other examples, progression of voice and speechimpairment is a symptom of Parkinson's disease, and progression ofsyntactic complexity impairment, forgetting of familiar words or thelocation of everyday objects, general problems with memory andconcentration, and trouble remembering new names are symptoms ofAlzheimer's disease.

The timely recognition and diagnosis of brain related illnesses andinjuries and neurological disorders and is critical for treatment to beadministered effectively. For example, the early detection and treatmentof ischemic stroke can limit brain damage and greatly improve apatient's outcome. As another example, because diagnoses of Alzheimer'sdisease are more accurate early in the disease's progression, earlydetection can allow for treatment to be more effective and in some casescan reverse some cognitive decline.

Early detection of a change or progression in someone's speech generallyrequires some knowledge of the person's previous “normal” speech. Afriend or family member who is in regular contact with a family membermay be able to notice a subtle change or progression in a familymember's speech. For example, as symptoms of Alzheimer's disease, afriend or family member may notice extended pauses in a person's speech(agnosia), or the forgetting of names etc. due to memory loss. Asanother example, as symptoms of Parkinson's disease, a friend or familymember may notice hoarseness, softer tones, imprecise articulation anddifferences in the rate of spoken words, and pauses between spokenwords.

However, if a person lives alone or rarely interacts with friends andfamily, a subtle change or progression in the person's speech can gounnoticed for a relatively long period of time such that the earlywarning of an illness or disorder can be limited. Similarly, where amedical professional such as a doctor is not in regular contact with thefamily member, a subtle change in the family member's speech may not benoticed by the doctor because they are unfamiliar with the person'sprevious “normal” speech such that the early warning of an illness ordisorder can be limited.

Even if a friend or family member or a medical professional who is inregular contact with a family member may be able to notice a subtlechange or progression in a family member's speech, the individual maynot represent their observations objectively or independently. Forexample, an individual may not accurately report or fully reveal anobserved change or progression in a person's speech to protect theperson for legal or employment purposes, or to protect the individual'sown personal interests.

Limited access to a recording of a person's “normal” or “baseline”speech can frustrate detection of a change or progression in theperson's speech. For example, a medical professional who is not inregular contact with a patient or who encounters the patient for thefirst time may not know how the patient's “normal” or previous speechactually sounds, and may not have access to or be able to listen to anaudio version of the recording of the patient's “normal” or previousspeech. Where there is no recording of the patient's speech to define“normal” or previous speech for that patient, the medical professionalmay not know how the speech of a person similarly situated in age,demographics, dialects, medical condition, etc. to the patient shouldgenerally sound. In this case, an audio version of the recording ofspeech of such similarly-situated person may be helpful to the medicalprofessional.

Limited access to a person's speech recordings which have been collectedover time can further limit analysis and detection of an illness, injuryor disorder and administration of a speech therapy therefor. Forexample, where different speech recordings are saved or stored invarious places, analysis of the speech recordings can be frustrated suchthat a stutter can go undetected and there is a missed opportunity forproviding exercises or treatment to the person for the stutter. Evenwhere the stutter is detected, the analysis may not prompt the person inreal-time to perform the exercises, especially in a non-clinicalenvironment such as in the person's home or regular living areas.

Turning now to an overview of the aspects of the invention, one or moreembodiments of the invention address the above-described shortcomings ofthe prior art by providing a NLP system and related methodologiesconfigured to cognitively analyze speech recordings to generateobjective classifications of an individual person's speech to be used asa baseline “normal” speech of the individual person, from which a changeor progression of a property or characteristic in the person's speechoutside of what is expected can be determined. Furthermore, the NLPsystems and methods in accordance with aspects of the invention use thecognitive analysis to diagnose an illness, injury or disorder and toprompt an individual to initiate exercises or treatment related to suchdiagnosis.

The above-described aspects of the invention address the shortcomings ofthe prior art by using machine learning algorithms to classify aperson's baseline “normal” speech based on cognitive analysis oftraining data that can include a recording of that person's speechand/or a recording of speech for one or more individuals similarlysituated in age, demographics, dialects, medical condition, etc. to theperson, including what can be expected in one's speech owing to factorssuch as aging, tooth loss or compromised dentition, and speechimpediments. Such classification of the person's baseline “normal”speech avoids subjective observation of the person's speech andcustomizes a “baseline” from which a change or progression in theperson's speech outside of what is expected can be determined.

In accordance with aspects of the invention, the NLP system can includea classifier (or classifier algorithm) configured and arranged to usemachine learning algorithms to apply machine learning techniques to theabove-described training data. In aspects of the invention, theclassifier (or classifier algorithm) uses the machine learningalgorithms to extract features from the training data in order to“classify” the training data and uncover relationships between and amongthe classified training data. The classifier uses the classifiedtraining data and the uncovered relationships between and among theclassified training data to create a model of the person's baselinenormal speech, which can be subsequently compared to the person's actualspeech to detect deviations from the baseline norm. Examples of suitableimplementations of the classifier and machine learning algorithms of theNLP system include but are not limited to neural networks, supportvector machines (SVMs), logistic regression, decision trees, hiddenMarkov Models (HMMs), etc. The learning or training performed by theclassifier can be supervised, unsupervised, or a hybrid that includesaspects of supervised and unsupervised learning. Supervised learning iswhen training data are already available and classified/labeled.Unsupervised learning is when training data are not classified/labeledso they must be developed through iterations of the classifier.Unsupervised learning can utilize additional learning/training methodsincluding, for example, clustering, anomaly detection, neural networks,deep learning, and the like.

In aspects of the invention, the classifier of the NLP system can beconfigured to apply confidence levels (CLs) aspects of the baselinenormal speech model. When the classifier determines that a CL in anaspect of the baseline normal speech model is below a predeterminedthreshold (TH) (i.e., CL<TH), the CL in that portion of the baselinenormal speech model can be classified as sufficiently low to justify aclassification of “no confidence” in that portion of the baseline normalspeech model, in which case, that portion of the baseline normal speechmodel would not be used until (and unless) its CL is improved throughadditional training data. If CL>TH, the CL in that portion of thebaseline normal speech model can be classified as sufficiently high tojustify using that portion of the baseline normal speech model to makecomparisons to the person's actual speech to detect deviations in theactual speech from the baseline normal model. Many differentpredetermined TH levels can be provided. The various portions of thebaseline normal speech model with CL>TH can be ranked from the highestCL>TH to the lowest CL>TH.

The above-described aspects of the invention further address theshortcomings of the prior art by saving and storing the person's actualspeech recordings and those of one or more individuals similarlysituated to the person, as well as the analysis of each of theserecordings in a relatively permanent fashion, using blockchaintechnology. Blockchain is a digital and decentralized ledger thatrecords information and transactions and can be used for collecting thespeech recordings as well as the analysis of each of these recordings.By design, a blockchain is resistant to modification of the data. It isan open, distributed ledger that can record transactions between twoparties efficiently and in a verifiable and permanent way. For use as adistributed ledger, a blockchain is typically managed by a peer-to-peernetwork collectively adhering to a protocol for inter-node communicationand validating new blocks. Once recorded, the data in any given blockcannot be altered retroactively without alteration of all subsequentblocks, which requires consensus of the network majority. Althoughblockchain records are not unalterable, blockchains may be consideredsecure by design and exemplify a distributed computing system with highByzantine fault tolerance.

In more detail, a blockchain is a growing list of records, calledblocks, which are linked using cryptography. Each block contains acryptographic hash of the previous block, a timestamp and transactiondata, etc. Blockchain technology offers a way for users to rely on acommon digital history which is important because digital informationand transactions are in theory easily faked and/or duplicated.Blockchain technology solves this problem without requiring a trustedintermediary. As such, by using this technology, users can confirm therecorded information and transactions without the need for a centralcertifying authority.

The use of blockchain technology facilitates continuous training andrefinement of a person's “baseline” (i.e., the baseline normal speechmodel) by providing the machine learning and cognitive computingtechniques with access to a relatively large sample of the person'sspeech recordings collected over time and the cognitive analysisthereof. Additionally, use of blockchain technology provides improvedaccess to the stored speech recordings and analysis thereof forreal-time retrieval and use by family members, caregivers, medicalprofessionals and even the actual person, such as through mobileelectronic communication devices like a smartphone, an electronicintelligent or virtual personal assistant such as the Amazon Alexa, etc.

Turning now to a more detailed description of aspects of the presentinvention, FIG. 1 depicts a system of speech detection and analysisaccording to an embodiment of the invention. FIG. 1 shows a person 100from which speech is input to a device 300. For example, the device 300can be a mobile communication device such as the person's smartphone oran intelligent or virtual personal assistant located in the person'snon-clinical private environment like a home or regular living area. Forexample, the device 300 can be an instrument in a doctor's office wherethe speech (or an interview) is recorded and can be compared to theprevious time the person visited the doctor. The device 300 includes anaudio input 350 which receives the speech of the person 100.

In one or more embodiments, the device 300 can be triggered by theperson's voice to collect speech of the person 100, while analysis ofthe collected speech occurs outside of the device 300 such as in a cloudcomputing environment. The device 300 can include a detector such as asensor which detects the presence of the person's voice to startrecording of the person's speech. For example, the device 300 can be anintelligent or virtual personal assistant located in the home or regularliving area of the person 100, where the intelligent or virtual personalassistant is turned on to sample the voice or speech characteristics ofthe person's speech when the person's voice is sensed.

In one or more embodiments, the device 300 could be programmed to turnon at fixed intervals to sample the voice or speech characteristicswithin speech of the person 100, while analysis of the collected speechoccurs outside of the device 300 such as in a cloud computingenvironment. For example, the device 300 can be a smartphone which isprogrammed to record the person's speech for a specified period of timeusing normal language or predefined sentences. By using an applicationon the smartphone, speech recording is turned on at specified timesduring a day or week to record the person's speech so that the person100 would not have to separately enable the application.

In one or more embodiments, the device 300 can store the collectedspeech. In one or more embodiments, the collected speech can be storedand analyzed by one or more of a computing or processing device 200. Thecomputing or processing device 200 is communicatively connected to thedevice 300. In accordance with aspects of the invention, the device 200can be implemented as a NLP system having machine learning algorithmsand classifier algorithms configured to perform operations in accordancewith aspects of the invention.

A signal such as an audio or visual signal can be output from the device300 in response to the information or results of the analysis from thecomputing or processing device 200. The signal can be provided to theperson 100 from a signal output 460 of the device 300.

Collected speech, analysis thereof, and information and results of theanalysis can be communicated to a user's device 400 from the computingor processing device 200. The computing or processing device 200 iscommunicatively connected to the user's device 400. For example, theuser of the device 400 can be a family member, a caregiver or a medicalprofessional participating in the care of the person 100. The user ofthe device 400 may not only view communicated information and results ofthe analysis of collected speech, but may also listen to an audioversion of actual collected speech such as through an audio signaloutput 460 of the user's device 400. In one or more embodiments, thedevice 300 and the user's device 400 can be the same device. Forexample, as the user's device 400, a family member, a caregiver or amedical professional participating in the care of the person 100 can usethe device 300 to access the baseline “normal” speech (i.e., thebaseline normal speech model) of a person 100 even including the audioversion thereof.

Collected speech, analysis thereof, and information and results of theanalysis, such as a diagnosis of an illness, injury or disorder, can bestored or maintained on the computer or computer processing device 200.Based on the information and results of the analysis, in one or moreembodiments, the computer or computer processing device 200 may takeaction by merely storing and maintaining the collected speech, theanalysis thereof, and the information and results of the analysis. Basedon the information and results of the analysis, in one or moreembodiments, the computer or computer processing device 200 may takeaction by communicating or disseminating the collected speech, theanalysis thereof, and the information and results of the analysis to oneor more of the person 100, a family member, a caregiver or a medicalprofessional participating in the care of the person 100. Based on theinformation and results of the analysis, in one or more embodiments, thecomputing or processing device 200 can initiate specific action relatedto the diagnosis of the illness, injury or disorder beyond merelystoring and maintaining or disseminating the collected speech, theanalysis thereof, and the information and results of the analysis.

In one or more embodiments, the computer or computer processing device200 can be an external server, such as in a cloud computing environment,which can interface with a variety of different devices. In one or moreembodiments, collected speech, analysis or comparison thereof, andinformation and results of the analysis and comparison, can be stored ormanaged in a relatively permanent fashion by using blockchaintechnology. The collected speech, analysis or comparison thereof, andinformation and results of the analysis and comparison, can be stored ormanaged in an electronically-accessible digital format and/or anelectronically-accessible audio format by using the blockchaintechnology.

FIG. 2 depicts a flowchart of a method of speech detection and analysisaccording to an embodiment of the invention. In one or more embodiments,the method of speech detection and analysis is a computer-implemented.

In 1000, for a person (100 in FIG. 1) whose speech is to be analyzed todiagnose a disorder, a baseline “normal” speech model is constructed orestablished. In one or more embodiments, the baseline “normal” speechmodel for a person is defined according to personal attributes of theperson such as age, population, demographics, geographic location,dialect, existing illness, injury or disorder, existing speechimpediment, natural progression of voice or speech (e.g., expected withaging due to tooth loss and compromised dentition, etc.) or acombination thereof.

The baseline “normal” speech model can define the absence or presence,the level, or rate of change of one or more property or characteristicin a person's speech.

In one or more embodiments, the baseline “normal” speech model caninclude the absence (e.g., zero level or zero degree) of a property orcharacteristic in a person's speech, outside of what is expected giventhe person's age, population, demographics, geographic location,dialect, existing illness, injury or disorder, existing speechimpediment, natural progression of voice or speech or combinationthereof. For example, after taking into account what speech is expectedfor a person, absence of aphasia, verbal apraxia, etc. which are commonindications of stroke can be considered “normal” speech.

In one or more embodiments, the baseline “normal” speech model caninclude some presence (e.g., greater than zero level or greater thanzero degree) of a property or characteristic in a person's speech,outside of what is expected, but still used as the baseline “normal”speech for a person. For example, for a first-ever recording of aperson's speech, there may already be some presence of a property orcharacteristic in the person's speech. While this presence may indicatean illness when compared to a threshold for that illness, the presenceof the property or characteristic in the person's speech also becomesthe person's baseline “normal” speech from which to compare futurespeech recordings in determining illness progression.

In one or more embodiments, the baseline “normal” speech model can beconstructed based on using machine learning algorithms to extractfeatures from the person's recorded speech, which inherently reflectsthe person's age, population, demographics, geographic location,dialect, existing illness, injury or disorder, existing speechimpediment or natural progression of voice or speech (e.g., expectedwith aging due to tooth loss and compromised dentition, etc.). Forexample, for a person who is diagnosed as having a stroke, the resultingverbal apraxia, etc. can become the baseline “normal” speech for thatperson from which any further changes in speech can be compared.

In one or more embodiments, the baseline “normal” speech model can bedefined by one or more individual's recorded speech and analysisthereof, where the individual is similarly-situated to the person suchas in age, population, demographics, geographic location, dialect,existing illness, injury or disorder, existing speech impediment ornatural progression of voice or speech. Such individual baseline canhereinafter be referred to as a “cohort” baseline because the person andthe similarly-situated individual are considered part of a group havingone or more common characteristics (e.g., age, population, demographics,geographic location, dialect, existing illness, injury or disorder,existing speech impediment or natural progression of voice or speech,etc.).

The baseline “normal” speech model described above, can be constructedusing automated machine learning and cognitive computing techniques thatcan extract and quantify relatively important metrics and propertiespresent in recorded speech. As an example of such techniques, latentsemantic analysis is a natural language processing technique thatperforms a high-dimensional associative analysis of semantic structureto detect common structures that occur in an individual's speechpatterns. As another example, a data-driven component extractionalgorithm such as independent component analysis can select relevantfeatures that consistently predict deviation from a baseline, from amongfeatures within recorded speech.

Different analysis and extractions within the automated machine learningand cognitive computing techniques can be performed on a same recordedspeech source. For example, spectrograms extracted from sound producedduring a speech recording can be automatically analyzed using variousinformation processing chains. For the same speech recording, syntacticand structural information can be automatically extracted from textproduced by a speech-to-text conversion, then analyzed for features suchas ‘coherence’, which are useful at discriminating neurological diseasefrom healthy speech.

By using blockchain technology to store or manage in a relativelypermanent fashion the collected speech, analysis thereof, andinformation and results of the analysis such as the “normal” baseline, abaseline of “normal” speech can be continuously refined because themachine learning and cognitive computing techniques provide an increasedaccess to a relatively large sample of the person's speech recordings orthe similarly-situated individual's speech recordings collected overtime and the analysis of each of these speech recordings. That is, thebaseline “normal” speech model is essentially continuously trained,updated and customized to the person whose speech is to be analyzed todiagnose a disorder such that even subtle changes or progression in theperson's speech can be objectively detected and analyzed.

In 1010, the speech of the person is compared to the baseline “normal”speech model established for the person. More particularly, thereal-time or current speech of the person is compared to thecorresponding baseline “normal” speech model established for thatperson. In one or more embodiments, the absence or presence, the leveland/or rate of change of one or more property or characteristic in thereal-time or current speech of the person is compared to those in the“normal” speech baseline established for that person, to determinedeviation from the “normal” baseline.

A deviation can represent a significant change beyond some threshold inthe person's ability to speak, for example, verbal apraxia in stroke.Or, the deviation can be more subtle, for example, the forgetting of apet's name, an important date, or the ability to recognize what anobject is or what it is used for (agnosia) as is a symptom ofAlzheimer's disease. For detection of symptoms typical of the mild stageof Alzheimer's disease, for example, deviations include the forgettingof familiar words, the location of everyday objects, general problemswith memory and concentration, and trouble remembering new names. Thedeviations can occur slowly over time as is the case with differentforms of dementia.

In one or more embodiments, the real-time or current speech can be arecording of the person's speech, such as the latest recording of theperson's speech.

In one or more embodiments, the real-time or current speech can be theactive or actual speech of a person observed by someone like a familymember, a caregiver or a medical professional participating in the careof the person. For example, a doctor who is located in the person'simmediate physical environment can be observing active speech of theperson during the person's visit to the emergency room or to thedoctor's office.

In one or more embodiments, the computer or computer processing device(200 in FIG. 1) performs the comparison of the real-time or currentspeech of the person to the corresponding baseline “normal” speechestablished for that person. In one or more embodiments, the computer orcomputer processing device can employ the machine learning and cognitivecomputing techniques described above to compare the real-time or currentspeech of the person to the corresponding baseline “normal” speechestablished for that person.

In one or more embodiments, someone like the family member, thecaregiver or the medical professional participating in the care of theperson can compare the real-time or current speech of the person to thecorresponding baseline “normal” speech established for that person. Forexample, whether located in the person's immediate physical environmentor located remotely from the person's immediate physical environment, adoctor which employs the user's device (400 in FIG. 1) described abovecan listen to an audio version of the actual speech recording of thebaseline “normal” speech for the person via the user's device, wheresuch speech recording is from the person's recorded speech or from asimilarly-situated individual's recorded speech. In one or moreembodiments, the doctor can listen to an audio version of one or both ofthe baseline “normal” speech recordings among the person's recordedspeech and the similarly-situated individual's recorded speech.

As similarly discussed above, by using blockchain technology to store ormanage in a relatively permanent fashion the actual speech recording ofthe baseline “normal” speech for the person, access to the baseline“normal” speech for the family member, the caregiver or the medicalprofessional comparing the real-time or current speech of the person tothe corresponding baseline “normal” speech established for that personcan be increased. By increasing access to the actual speech recording ofthe baseline “normal” speech for the person, the family member, thecaregiver or the medical professional can have an objective example of aperson's “normal” speech from which even subtle changes or progressionin the person's real-time or current speech can be observed.Furthermore, using blockchain would allow the recorded speech anddeviations to be stored in a relatively permanent fashion which could beuseful for a family member or a caregiver or for a medical professionalin the case of a lawsuit.

In 1020, as information or results of the above-described comparison, adeviation between the real-time or current speech of the person and thebaseline “normal” speech model established for the person is determined.In one or more embodiments, the computer or computer processing devicewhich performs the comparison of the real-time or current speech of theperson to the corresponding baseline “normal” speech established forthat person determines the deviation. The computer or computerprocessing device can employ the machine learning and cognitivecomputing techniques described above to determine the deviation betweenthe real-time or current speech of the person and the correspondingbaseline “normal” speech established for that person.

In one or more embodiments, the family member, the caregiver or themedical professional who performs the comparison of the real-time orcurrent speech of the person to the corresponding baseline “normal”speech established for that person determines the deviation.

In one or more embodiments, the deviation can be expressed by adifference in the absence or presence, the level, or rate of change ofone or more property or characteristic between the real-time or currentspeech of the person and the corresponding baseline “normal” speechestablished for that person.

In one or more embodiments, the deviation from the comparison of thereal-time or current speech of the person to the corresponding baseline“normal” speech established for that person, can be stored or maintainedon the computer or computer processing device. In one or moreembodiments, speech could be recorded in the block (e.g., withinBlockchain technology) when the deviation exceeds a threshold and therate of recording in the block could be connected to the rate at whichthe speech deviates from the threshold.

As similarly discussed above, by using blockchain technology to store ormanage in a relatively permanent fashion the collected speech, analysisand comparison thereof, and information and results of the analysis andcomparison such as deviation from “normal” baseline, a baseline “normal”speech can be continuously refined because the machine learning andcognitive computing techniques are provided with increased access to arelatively large sample of the person's speech recordings or thesimilarly-situated individual's speech recordings collected over timeand the analysis of each of these speech recordings. Again, the baseline“normal” speech is essentially continuously updated and customized tothe person whose speech is to be analyzed by a medical professional.Furthermore, using blockchain technology would allow the recorded speechand deviations to be stored in a relatively permanent fashion whichcould be useful for a family member or a caregiver or for a medicalprofessional in the case of a lawsuit.

In 1030, the deviation is compared to an established threshold todetermine if the deviation meets or exceeds the established threshold.In one or more embodiments, the computer or computer processing devicewhich determines the deviation compares the deviation to the establishedthreshold. In one or more embodiments, the family member, the caregiveror the medical professional which determines the deviation compares thedeviation to the established threshold.

A different threshold could be established for different medicalconditions, for different personal attributes, for different “cohort”groups in which individuals have one or more common characteristic, etc.The threshold can be expressed in terms of the absence or presence, thelevel, the rate of change of one or more property or characteristic in aperson's speech, differences in any one of the absence or presence, thelevel, the rate of change of one or more property or characteristic, ora deviation from an established threshold of the absence or presence,the level, the rate of change of one or more property or characteristic.For example, a change to presence of aphasia, verbal apraxia, etc. fromabsence thereof can define a threshold for abnormal speech relative to astroke. For example, a change of the absence or presence, the level, therate of change of one or more property or characteristic in a person'sspeech, which is expected due to an existing speech impediment ornatural progression of voice or speech (e.g., due to aging, tooth loss,compromised dentition, etc.) may not define a threshold for abnormalspeech relative to an illness, injury or disorder such as stroke. Forexample, the deviation from an established threshold of a property orcharacteristic in speech of a “cohort” individual similarly-situated toa person may define a threshold for abnormal speech relative to dementiaor may merely define a threshold for normal aging.

In one or more embodiments, the deviation defined for a same real-timeor current speech of a person can be compared to one or more establishedthreshold at substantially a same time. A deviation of a property orcharacteristic in a person's speech which is beyond a threshold for oneillness, injury or disorder may not be beyond a threshold for anotherillness, injury or disorder.

As similarly discussed above, by using blockchain technology which isresistant to modification to store or manage in a verifiable andrelatively permanent fashion the deviation from “normal” baseline,efficient comparison of the deviation to multiple thresholds can beperformed such that subtle changes or progression in the person's speechfor a number of illnesses, injuries or disorders can be objectivelydetected.

In one or more embodiments, if the threshold is met or exceeded, thecomputer or computer processing device initiates action directly relatedto the illness, injury or disorder.

In one or more embodiment where the threshold is met or exceeded, thecomputing or processing device (200 in FIG. 1) can initiate notificationof a doctor, emergency services or a family member or could call a helpline, etc. depending on the severity and rate of change of a property orcharacteristic in a person's speech.

In one or more embodiment where the threshold is met or exceeded, thecomputing or processing device (200 in FIG. 1) can adjust frequency ofturn on-turn off times of the device (100 in FIG. 1), toincrease/changing time of day for collection, especially wheredeviations from the “normal” baseline exceed a threshold. For example,the rate of recording by the device could be adjusted to correspond tothe rate at which the speech deviates from the threshold.

In one or more embodiment where the threshold is met or exceeded, thecomputing or processing device (200 in FIG. 1) can prompt proactivetreatment related to such diagnosis. For example, the computing orprocessing device can provide prompts to the person via the device (100or 400 in FIG. 1) to elicit speech from the person. The prompts caninclude interactive exercises that specifically use tests for memory andconcentration or are designed for treatment related to an illness,injury or disorder such as speech impediments due to a stutter. Forexample, the prompts can include recorded speech which is played back inan audio version to facilitate treatment of the stutter. Providing theprompt for proactive treatment related would further allow refinement ofthe baseline “normal” speech of the person by highlighting deviationswhich exist due to stutters, for example, and are not abnormaldeviations that should trigger a notification for diagnosis of a moreserious illness, injury or disorder.

In one or more embodiment where the deviation does not meet or exceedthe threshold, the computer or computer processing device 200 may notinitiate action directly related to the illness, injury or disorder. Forexample, the last-recorded speech of the person can be merely saved as anew baseline “normal” speech established for the person from which tocompare a later-recorded speech of the person.

FIG. 3 depicts the major hardware components of the computer or computerprocessing device (200 in FIG. 1) according to an embodiment of thepresent invention. In one or more embodiments, the computer or computerprocessing device stores and/or manages collected recorded speech,analysis and comparison thereof, and information and results of theanalysis and comparison by using blockchain technology.

A programmable processor 210 executes a computer control program 220 tostore and/or manage the collected recorded speech, analysis andcomparison thereof, and information and results of the analysis andcomparison.

The analysis module 230 performs analysis of the collected recordedspeech, such as by machine learning and cognitive computing techniques.The analysis module 230 can construct and refine a “normal” baselinecorresponding to a person (1000 in FIG. 2), compare real-time or currentspeech of the person to the corresponding baseline “normal” speech (1010in FIG. 2), determine a deviation from the comparison between thereal-time or current speech of the person and the corresponding baseline“normal” speech (1020 in FIG. 2) and determine if the deviation meets athreshold (1030 in FIG. 2).

The memory 240 stores the recorded collected speech, analysis andcomparison thereof, and information and results of the analysis andcomparison which are variously generated in 1000 through 1030 of FIG. 2.Collected recorded speech, prompts such as interactive exercises, etc.are variously communicated between the computer or computer processingdevice and other devices connected thereto, via a wireless node 250,which can include any sort of remote connection. The wireless node 250allows the computer or computer processing device to be connected toother devices such as the device (300 in FIG. 1) and the user's device(400 in FIG. 1). In other embodiments of the present invention,traditional wire systems can be used. The computer or computerprocessing device can include a power source 260 which can be a battery,electric feed, or any other method known in the relevant art.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instruction by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdescribed herein.

What is claimed is:
 1. A natural language processing system foranalyzing speech, the system comprising: a computer processing deviceconfigured to receive recorded speech of a person, wherein the computerprocessing device: constructs a baseline speech model of the person, thebaseline speech model of the person comprising a property of speechbased on a personal attribute of the person, compares current recordedspeech of the person to the baseline speech model of the person todetermine a deviation of the property of speech therebetween, anddetermines if the deviation of the property of speech meets a thresholdof the property of speech that is defined for a disorder.
 2. The systemof claim 1, wherein the computer processing device further stores therecorded speech of the person and the baseline speech model of theperson using blockchain technology.
 3. The system of claim 2, whereinthe computer processing device further stores an electronicallyaccessible audio version of the baseline speech model of the person byusing the blockchain technology.
 4. The system of claim 1, wherein thepersonal attribute of the person comprises one or more of age,population, demographics, dialect, an existing illness, injury, disorderand speech impediment, and natural progression of voice that is due toany of the aforementioned personal attributes.
 5. The system of claim 1,wherein the computer processing device constructs the baseline speechmodel of the person by using machine learning and cognitive computingtechniques with which the property of speech is extracted and quantifiedfrom a speech recording.
 6. The system of claim 5, wherein: the speechrecording comprises recorded speech of an individual person having acommon personal attribute with the person, and the property of speechbased on the personal attribute of the person within the baseline speechmodel of the person is defined from the analyzed recorded speech of theindividual person having the common personal attribute with the person.7. The system of claim 5, wherein: the speech recording comprises aplurality of recorded speeches of the person that are taken over aperiod of time; and the property of speech based on the personalattribute of the person within the baseline speech model of the personis defined from the analyzed plurality of recorded speeches of theperson.
 8. The system of claim 1, wherein the computer processing devicethat determines that the deviation of the property of speech meets thethreshold of the property of speech that is defined for the disorder,further determines a communication directly related to the disorder, andthe communication directly related to the disorder comprises aninteractive exercise that elicits additional recorded speech from theperson to provide speech therapy to the person.
 9. The system of claim8, wherein the disorder for which the speech therapy is provided to theperson comprises one or more of: a speech impediment comprisingstuttering, and a neurological disorder or brain-related disordercomprising one or more of stroke, Alzheimer's disease, Parkinson'sdisease and dementia.
 10. The system of claim 1, wherein the computerprocessing device that determines that the deviation of the property ofspeech meets the threshold of the property of speech that is defined forthe disorder, further determines a communication directly related to thedisorder, and the communication directly related to the disordercomprises the diagnosis of a neurological disorder, brain-relatedillness or brain-related injury of the person.
 11. The system of claim1, wherein the property of speech comprises one or more of cadence,tone, pitch, speed, word content, word complexity, pauses, periodicity,volume, semantic structure, sound produced during speech and coherency.12. A computer-implemented method that processes natural language foranalyzing speech, the computer-implemented method comprising: receivingrecorded speech of a person; constructing a baseline speech model of theperson, the baseline speech model of the person comprising a property ofspeech based on a personal attribute of the person; comparing currentrecorded speech of the person to the baseline speech model of the personto determine a deviation of the property of speech therebetween; anddetermining if the deviation of the property of speech meets a thresholdof the property of speech that is defined for a disorder.
 13. Thecomputer-implemented method of claim 12 further comprising storing therecorded speech of the person and the baseline speech model of theperson using blockchain technology.
 14. The computer-implemented methodof claim 12, wherein the constructing the baseline speech model of theperson comprises using machine learning and cognitive computingtechniques with which the property of speech is extracted and quantifiedfrom a speech recording.
 15. The computer-implemented method of claim14, wherein the speech recording comprises recorded speech of anindividual person having a common personal attribute with the person,and the property of speech based on the personal attribute of the personwithin the baseline speech model of the person is defined from theanalyzed recorded speech of the individual person having the commonpersonal attribute with the person.
 16. The computer-implemented methodof claim 14, wherein the speech recording comprises a plurality ofrecorded speeches of the person that are taken over a period of time,and the property of speech based on the personal attribute of the personwithin the baseline speech model of the person is defined from theanalyzed plurality of recorded speeches of the person.
 17. Thecomputer-implemented method of claim 12, wherein determining that thedeviation of the property of speech meets the threshold of the propertyof speech that is defined for the disorder comprises determining acommunication directly related to the disorder, and the communicationdirectly related to the disorder comprises an interactive exercise thatelicits additional recorded speech from the person to provide speechtherapy to the person.
 18. The computer-implemented method of claim 12,wherein determining that the deviation of the property of speech meetsthe threshold for the property of speech that is defined for thedisorder comprises determining a communication directly related to thedeviation of the property, and the communication directly related to thedeviation of the property comprises the baseline speech model of theperson, the deviation of the current recorded speech of the person fromsuch baseline speech model, and the threshold of the property of speechwhich is met by such deviation.
 19. A computer program product whichprocesses natural language for analyzing speech, the computer programproduct comprising a computer readable storage medium having programinstructions embodied therewith, wherein the program instructions areexecutable by a computer to cause the computer to perform a methodcomprising: receiving recorded speech of a person; constructing abaseline speech model of the person, the baseline speech model of theperson comprising a property of speech based on a personal attribute ofthe person; comparing current recorded speech of the person to thebaseline speech model of the person to determine a deviation of theproperty of speech therebetween; and determining if the deviation of theproperty of speech meets a threshold of the property of speech that isdefined for a disorder.
 20. The computer program product of claim 19,wherein the method further comprises storing the recorded speech of theperson and the baseline speech model of the person using blockchaintechnology.